Deconstructing Reinforcement Learning in Sigma
نویسنده
چکیده
This article describes the development of reinforcement learning within the Sigma graphical cognitive architecture. Reinforcement learning has been deconstructed in terms of the interactions among more basic mechanisms and knowledge in Sigma, making it a derived capability rather than a de novo mechanism. Basic reinforcement learning – both model-based and model-free – are demonstrated, along with the intertwining of model learning.
منابع مشابه
Deconstructing Episodic Memory and Learning in Sigma
In an experiment in functional elegance, episodic memory and learning have been deconstructed in the Sigma cognitive architecture in terms of pre-existing memory and learning mechanisms plus a template-based structure generator. As a side effect, base-level activation also becomes deconstructed in terms of a learned temporal prior.
متن کاملToward a Neural-Symbolic Sigma: Introducing Neural Network Learning
Building on earlier work extending Sigma’s mixed (symbols + probabilities) graphical band to inference in feedforward neural networks, two forms of neural network learning – target propagation and backpropagation – are introduced, bringing Sigma closer to a full neural-symbolic architecture. Adapting Sigma’s reinforcement learning (RL) capability to use neural networks in policy learning then y...
متن کاملDouble Q($\sigma$) and Q($\sigma, \lambda$): Unifying Reinforcement Learning Control Algorithms
Temporal-difference (TD) learning is an important field in reinforcement learning. Sarsa and Q-Learning are among the most used TD algorithms. The Q(σ) algorithm (Sutton and Barto (2017)) unifies both. This paper extends the Q(σ) algorithm to an online multi-step algorithm Q(σ, λ) using eligibility traces and introduces Double Q(σ) as the extension of Q(σ) to double learning. Experiments sugges...
متن کاملA Nonlinear Noise-Shaping Delta-Sigma Modulator with On-Chip Reinforcement Learning∗
The stability and quality of noise shaping is a concern in the design of higher-order delta-sigma modulators for oversampled analog-to-digital conversion. We reformulate noise-shaping modulation alternatively as a nonlinear control problem, where the objective is to find the binary modulation sequence that minimizes signal swing in a cascade of integrators operating on the difference between th...
متن کاملTowards Adaptive, Interactive Virtual Humans in Sigma
Sigma is a nascent cognitive architecture/system that combines concepts from graphical models with traditional symbolic architectures. Here an initial Sigma-based virtual human (VH) is introduced that combines probabilistic reasoning, rule-based decision-making, Theory of Mind, Simultaneous Localization and Mapping and reinforcement learning in a unified manner. This nonmodular unification of d...
متن کامل